Skip to content

[Holistic] TFTransformer#78

Closed
phi-dbq wants to merge 103 commits intodatabricks:masterfrom
phi-dbq:tf-transformer-part4
Closed

[Holistic] TFTransformer#78
phi-dbq wants to merge 103 commits intodatabricks:masterfrom
phi-dbq:tf-transformer-part4

Conversation

@phi-dbq
Copy link
Contributor

@phi-dbq phi-dbq commented Nov 18, 2017 •
edited
Loading

This PR provides a holistic view of the changes brought by TFTransformer which takes TensorFlow graphs and create MLlib Transformers out of them.

This PR contains all the commits in the four-part PR stack.

  • ✅ Part-1 Params and Converters
  • ✅ Part-2 TensorFlow Utility Extension
  • 🔄 Part-3 TFInputGraph
    • Sync'ed with @thunterdb offline and merged requested test refactor.
  • 🔄 Part-4 TFTransformer
    • Sync'ed with @thunterdb offline, authored and merged requested test refactor.

Please either follow either merge strategy after review, or make other changes as needed.

Merge Strategies

phi-dbq added 30 commits September 7, 2017 13:18
@phi-dbq
flat param API impl
42c6e6e
@phi-dbq
support input graph scenarios
ecbefb9
@phi-dbq
(WIP) new interface implementation
ab89bd2
@phi-dbq
docs and cleanup
8c7d72e
@phi-dbq
using tensorflow API instead of our utilities
eb543c6
@phi-dbq
automatic type conversion
4743bb9
@phi-dbq
cleanup
622c788
@phi-dbq
PR comments
07f1cec 
1. Move `InputGraph` to its module.
@phi-dbq
(WIP) address comments
692b0eb
@phi-dbq
(WIP) respond to PR comments
66d44e9
@phi-dbq
test refactor
9b3fe86
@phi-dbq
Merge remote-tracking branch 'upstream/master' into tf-1d-transformer
8c32501
@phi-dbq
(wip) consolidating params
dbd9aaa
@phi-dbq
rebase upstream
4572205
@phi-dbq
import params fix
1cc7591
@phi-dbq
(wip) TFInputGraph impl
2fc6787
@phi-dbq
(wip) moving to new API
889df0a
@phi-dbq
(wip) enable saved_model tests
86cd6d9
@phi-dbq
(wip) enable checkpoint test
ac09182
@phi-dbq
(wip) enable multiple tensor tests
6b22eed
@phi-dbq
enable all tests
a3517d6
@phi-dbq
params and converters
457a4c2
@phi-dbq
tests
323939a
@phi-dbq
Merge branch 'tf-transformer-part1' into api-tf-transformer
6e46073
@phi-dbq
optimize graph for inference
b232b3c
@phi-dbq
more tests
d921366
@phi-dbq
update utils
0c8c219
@phi-dbq
tests
522279a
@phi-dbq
intro: TFInputGraph
f4d938c
@phi-dbq
tests
cd3aa8d
phi-dbq and others added 20 commits October 2, 2017 19:01
@phi-dbq
Merge branch 'tf-transformer-part1' into tf-transformer-part3
7f16396
@phi-dbq
(wip) PR comments
f5107ad
@phi-dbq
more tests
ac681b0
@phi-dbq
change test generator module name
4d173c5
@phi-dbq
Merge branch 'tf-transformer-part3' into tf-transformer-part4
85e0778
@phi-dbq
tensor tests
22754c9
@phi-dbq
tensor test update
0144b8c
@phi-dbq
buildCheckList name change and doc fixup
a8531ec
@phi-dbq
Merge branch 'tf-transformer-part1' into tf-transformer-part2
3c849f2
@phi-dbq
Merge branch 'tf-transformer-part2' into tf-transformer-part3
707697d
@phi-dbq
Merge branch 'tf-transformer-part3' into tf-transformer-part4
c6eb87c
@phi-dbq
PR comments
d729528
@phi-dbq
PR comments
0c2eda1
@phi-dbq
PR comments
63967b4
@phi-dbq
Merge branch 'tf-transformer-part2' into tf-transformer-part3
fe719b2
@phi-dbq
Merge branch 'tf-transformer-part3' into tf-transformer-part4
812f4d6
@phi-dbq
TFTransformer Part-4 Test Refactor (#15)
47d497c 
* adding new tests

* remove original test design

* cleanup
@thunterdb @phi-dbq
TFTransformer Part-3 Test Refactor (#14)
a39b6d3 
* profiling

* tests

* renamed test

* removed original tests

* removed the profiler utils

* fixes indents

* imports

* added some tests

* added test

* fix test

* one more test
@phi-dbq
deleting original testing ideas
07cc335
@phi-dbq
Merge branch 'tf-transformer-part3' into tf-transformer-part4
925fc0d
@codecov-io
Copy link

codecov-io commented Nov 18, 2017 •
edited
Loading

Codecov Report

Merging #78 into master will increase coverage by 1.7%.
The diff coverage is 93.82%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master      #78     +/-   ##
=========================================
+ Coverage   83.93%   85.64%   +1.7%     
=========================================
  Files          26       29      +3     
  Lines        1407     1651    +244     
  Branches       17       17             
=========================================
+ Hits         1181     1414    +233     
- Misses        226      237     +11
Impacted Files Coverage Δ
python/sparkdl/transformers/named_image.py 93.51% <ø> (ø) ⬆️
python/sparkdl/transformers/keras_applications.py 85.71% <ø> (ø) ⬆️
python/sparkdl/param/image_params.py 74.28% <ø> (ø) ⬆️
python/sparkdl/transformers/tf_tensor.py 100% <100%> (ø)
python/sparkdl/transformers/keras_image.py 100% <100%> (ø) ⬆️
python/sparkdl/param/__init__.py 100% <100%> (ø) ⬆️
python/sparkdl/graph/builder.py 93.75% <100%> (+0.05%) ⬆️
python/sparkdl/graph/tensorframes_udf.py 90% <100%> (ø) ⬆️
python/sparkdl/graph/utils.py 100% <100%> (+4.93%) ⬆️
python/sparkdl/__init__.py 100% <100%> (ø) ⬆️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f088de4...af95b74. Read the comment docs.

@phi-dbq phi-dbq force-pushed the tf-transformer-part4 branch from 6687eab to 95cf964 Compare November 22, 2017 04:33
@phi-dbq phi-dbq force-pushed the tf-transformer-part4 branch from 95cf964 to af95b74 Compare November 22, 2017 04:34
@thunterdb
Copy link
Contributor

thunterdb commented Nov 22, 2017

Closing since #49 got merged.

@thunterdb thunterdb closed this Nov 22, 2017
@smurching smurching mentioned this pull request Dec 1, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@thunterdb thunterdb

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@phi-dbq @codecov-io @thunterdb